Video preview creation with audio

ABSTRACT

Providing a method for creating and displaying portions of videos called video previews. The video previews may be associated with audio, such that when the video previews are activated, the audio may play with the video preview. When multiple video previews are organized to play as a playable group or composite of video previews, a corresponding composite audio file can play in response to an activation of the composite.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/937,557, filed on Nov. 10, 2015, entitled “Video Preview Creationwith Audio”, which is a divisional of U.S. patent application Ser. No.14/173,740, filed on Feb. 5, 2014, entitled “Video Preview Creation withAudio.” U.S. patent application Ser. No. 14/173,740 is a non-provisionalof 61/761,096, filed on Feb. 5, 2013, U.S. Patent Application No.61/822,105, filed on May 10, 2013, U.S. Patent Application No.61/847,996, filed on Jul. 18, 2013, and U.S. Patent Application No.61/905,772, filed on Nov. 18, 2013, which are herein incorporated byreference in their entirety for all purposes.

This application is related to commonly owned U.S. patent applicationSer. No. 14/173,697, filed on Feb. 5, 2014, entitled “Video PreviewCreation with Link” (Attorney Docket 91283-000710US-896497), U.S. patentapplication No. 14/173,715, filed on Feb. 5, 2014, entitled “UserInterface for Video Preview Creation” (Attorney Docket91283-000720US-897301), U.S. patent application Ser. No. 14/173,732,filed Feb. 5, 2014, entitled “Video Preview Creation based onEnvironment” (Attorney Docket 91283-000730US-897293), U.S. patentapplication Ser. No. 14/173,745, filed Feb. 5, 2014, entitled“Generation of Layout of Videos” (Attorney Docket91283-000750US-897295), U.S. patent application Ser. No. 14/173,753,filed Feb. 5, 2014, entitled “Activating a Video Based on Location inScreen” (Attorney Docket 91283-000760US-897296), which are hereinincorporated by reference in their entirety for all purposes.

BACKGROUND

Users commonly provide video content to websites (e.g., YouTube®), whichcan be referred to as “posting a video.” The user can spend asignificant amount of time to convey the message of the video before theuser selects the video (e.g., by clicking the video displayed on awebsite). For example, the user can associate a title, a staticthumbnail image, and/or a textual description with the video. Usersoften have a difficult time when the video originates on a differentwebsite and the user tries to upload their video to a video server.Further, the title may not be descriptive of the contents of the video,the static thumbnail image may not summarize the essence of the video,or the description of the video may be a poor signal for whether thevideo will be interesting to a viewer.

Video browsing is also limited. Other users (e.g., viewers) can accessand view the video content via the websites. For example, the viewerscan see a video's title and static thumbnail of the video beforedeciding whether to play the full video. However, the viewers may findit difficult to select particular videos of interest because the titlemay not be descriptive of the contents of the video, the staticthumbnail image may not summarize the essence of the video, or thetextual description with the video may be a poor signal for whether thevideo will be interesting to the viewer. Thus, the viewers may spendsignificant amounts of time searching and watching videos that are notenjoyable to the viewer.

SUMMARY

Embodiments of the present invention can create and display portions ofvideos as video previews. The video previews may be associated with afull video, such that the video preview is generated from a portion ofthe full video. In some embodiments, the video preview can be configuredto play a series of images from the full video when the video preview isactivated.

In some embodiments, the video preview is combined with audio, such thatthe audio is played according to an audio rule (e.g., the audio rule candefine when to adjust the duration, pitch, volume, or fading in/out ofan audio file). When multiple video previews are organized to play as aplayable group or composite of video previews, a corresponding compositeaudio file can play in response to an activation of the composite.

Other embodiments are directed to systems and computer readable mediaassociated with methods described herein.

A better understanding of the nature and advantages of the presentinvention may be gained with reference to the following detaileddescription and the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a flowchart illustrating a method of creating a videopreview, organizing the video previews, and providing a user interfacethat includes the video previews according to an embodiment of thepresent invention.

FIG. 2 shows block diagrams of various subsystems used to generate orprovide a video preview.

FIG. 3 shows a flowchart illustrating a method of generating a videopreview with audio according to an embodiment of the present invention.

FIG. 4 shows a graphical user interface illustrating a video preview andone or more audio files according to an embodiment of the presentinvention.

FIG. 5 shows a graphical user interface illustrating a composite ofvideo previews according to an embodiment of the present invention.

FIG. 6 shows a flowchart illustrating a method of browsing videopreviews with audio according to an embodiment of the present invention.

FIG. 7 shows a flowchart illustrating a method of browsing videopreviews with audio according to an embodiment of the present invention.

FIG. 8 shows a graphical user interface illustrating an application thatdisplays video previews with audio according to an embodiment of thepresent invention.

FIG. 9 shows a block diagram of a computer apparatus according to anembodiment of the present invention.

DEFINITIONS

A “video preview” is a visual representation of a portion of a video(also referred to as a “full video” to contrast a “video preview” of thevideo). The full video may correspond to the entirety of a video file ora portion of the video file, e.g., when only a portion of the video filehas been streamed to a user device. The preview is shorter than the fullvideo, but the full video can be shorter than the complete video file.The preview can convey the essence of the full video. The video previewis shorter (e.g., fewer images, less time) than a full (e.g., moreimages, longer time, substantially complete) video. In variousembodiments, a preview can be a continuous portion of the full video orinclude successive frames that are not continuous in the full video(e.g., two successive frames of the preview may actually be one or moreseconds apart in the full video).

A “audio-video object” or “composite audio-video object” (usedinterchangeably) is an object on a GUI that is configured to play avideo previews (e.g., an iframe, a frame in a current window, framebuffer object (FBO)). An audio-video object or composite audio-videoobject (e.g., placeholders, 2-dimensional boxes, windows, or squares)can be generated by a computing device for displaying the video preview.In some embodiments, the audio-video object or composite audio-videoobject will also provide filters or effects for the video preview (e.g.,defined by the computing device, defined by a programming language thatgenerates an object class).

A “composite of video previews” (also referred to simply as a“composite” or “preview composite”) is an area where one or more relatedvideo preview blocks will be played. If the composite has one videopreview and/or one audio file, then the composite simply corresponds toplaying the preview, and the block and composite can correspond to thesame area. When the composite includes multiple previews and/or multipleaudio files, the composite includes multiple blocks, each playing apreview. The video previews in a composite can each link to the samefull video. In one embodiment, the creator of a preview can identify thepreviews or audio files to include in a composite, and the composite canexist as a single display object, where the previews of the compositestart playing at the time the composite is activated. The shape of acomposite of video previews can be a square (2 blocks×2 blocks, Nblocks×N blocks) or rectangle (1×N, 2×N, 2×2 with blocks comprisingunequal sides, N×N with blocks comprising unequal sides). The compositeof video previews may have a right-wise or left-wise orientation.

DETAILED DESCRIPTION

Embodiments of the present invention can enhance video viewing byproviding short, playable video previews through a graphical userinterface (GUI). Viewers can use the GUI of video previews to betterdecide whether to watch a full video, or channel of videos.

In one embodiment, the user may create a video preview that may later beaccessed by a viewer. For example, the user may select the best 1-10seconds of a video to convey the essence of the full video. The videopreview can be shorter (e.g., fewer images, less time) than a full(e.g., more images, longer time, substantially complete) video. Thesystem associated with the GUI may generate a smaller file to associatewith the video portion (e.g., animated GIF, MP4, collection of frames,RIFF). The system may provide the GUI on a variety of systems. Forexample, the GUI can be provided via an internet browser or clientapplications (e.g., software configured to be executed on a device), andconfigured to run on a variety of devices (e.g., mobile, tablet,set-top, television).

I. Providing Video Reviews

FIG. 1 shows a flowchart illustrating a method 100 of creating a videopreview, organizing the video previews, and providing a user interfacethat includes the video previews according to an embodiment of thepresent invention. The method 100 may comprise a plurality of steps forimplementing an embodiment of creating a video preview with a durationof a predetermined length of time. Various computing devices may be usedto perform the steps of the method, including video servers, providerservers, user devices, or third party servers.

At step 110, a video preview may be generated. Embodiments of theinvention may provide a method of generating an audio-video object thatincludes a video preview and an audio file. The audio file may playaccording to an audio rule that specifies one or more criteria for howthe audio file is to be played when the video preview is played inresponse to an audio-video object being activated. The video preview maybe created to convey the essence of the full video (e.g., the user mayselect the best 1-10 seconds of a video, like a building imploding or asnowboarder jumping off of a cliff).When multiple video previews areorganized to play as a playable group or composite of video previews, acorresponding composite audio file can play in response to an activationof the composite.

Additional means of generating video previews can be found in U.S.patent application Ser. No. 14/173,697, entitled “Video Creation withLink” (Attorney Docket 91283-000710US-896497), U.S. patent applicationSer. No. 14/173,715, entitled “User Interface for Video PreviewCreation” (Attorney Docket 91283-000720US-897301), and U.S. patentapplication Ser. No. 14/173,732, entitled “Video Preview Creation basedon Environment” (Attorney Docket 91283-000730US-897293), which areincorporated by reference in their entirety.

At step 120, one or more video previews may be organized into one ormore channels or collections. For example, the method 100 can associatethe video preview generated in step 110 (e.g., a 4-second animated GIFof a snowboarder jumping off a ledge) with a channel (e.g., a collectionof videos about snowboarders). In some embodiments, the video previewsmay be organized in a group (e.g., a composite, a playable group, acluster of video previews) and displayed on a network page. Additionalinformation about the organization and layout of video previews can befound in U.S. patent application Ser. No. 14/173,745, entitled“Generation of Layout of Videos” (Attorney Docket91283-000750US-897295), which is incorporated by reference in itsentirety.

At step 130, a GUI may be provided with the video previews. For example,the GUI may provide one or more channels (e.g., channel relating tosnowboarders, channel relating to counter cultures), one or more videoswithin a channel (e.g., a first snowboarding video, a secondsnowboarding video, and a first counter culture video), or a networkpage displaying one or more video previews. The video previews may beshared through social networking pages, text messaging, or other means.Additional information about viewing and sharing video previews can befound in U.S. patent application Ser. No. 14/173,753, entitled“Activating a Video Based on Location in Screen” (Attorney Docket91283-000760US-897296), which is incorporated by reference in itsentirety.

II. System for Providing Video Previews

Various systems and computing devices can be involved with variousworkflows used to create and play a video preview with audio.

FIG. 2 shows block diagrams of various subsystems used to generate orprovide a video preview. For example, the computing devices can includea video server 210, a provider server 220, a user device 230, or a thirdparty server 240 according to an embodiment of the present invention. Insome embodiments, any or all of these servers, subsystems, or devicesmay be considered a computing device.

The computing devices can be implemented various ways without divertingfrom the essence of the invention. For example, the video server 210 canprovide, transmit, and store full videos and/or video previews (e.g.,Ooyala®, Brightcover®, Vimeo®, YouTube®, CNN®, NFL®, Hulu®, Vevo®). Theprovider server 220 can interact with the video server 210 to providethe video previews. In some embodiments, the provider server 220 canreceive information to generate the video preview (e.g., a timestampidentifying a portion of a full video, a link to the full video, thefull video file, a push notification including the link to the fullvideo). The user device 230 can receive a video preview and/or fullvideo to view, browse, or store the generated video previews. The thirdparty server 240 can also receive a video preview and/or full video toview or browse the generated video previews. In some embodiments, theuser device 230 or third party server 240 can also be used to generatethe video preview or create a frame object (e.g., including anaudio-video object or a composite audio-video object). Additionalinformation about the video server 210, provider server 220, user device230, and third party server 240 can be found in U.S. patent applicationSer. No. 14/173,697, entitled “Video Preview Creation with Link”(Attorney Docket 91283-000710US-896497) and U.S. patent application Ser.No. 14/173,715, entitled “User Interface for Video Preview Creation”(Attorney Docket 91283-000720US-897301), which are incorporated byreference in their entirety.

In some embodiments, the video server 210, provider server 220, a userdevice 230, and third party server 240 can be used to generate anaudio-video object. For example, the computing device (e.g., providerserver 220, user device 230) may receive a video preview that comprisesone or more images. The images may be associated with a video portion ofthe full video that corresponds to a series of images from the fullvideo. The video can also have a video start and a video end. Thecomputing device can also receive an audio file (that has an audio startand an audio end) and an audio rule. The audio rule can specify one ormore criteria for how the audio file is to be played when the videopreview is played in response to an audio-video object being activated(e.g., in response to moving to a particular location on the display).The computing device can generate an audio-video object for displayingthe video preview and playing the audio file based on the audio rule.

In some embodiments, the playing of at least one of the audio start andthe audio end of the audio file is not synchronized with playing thevideo start and the video end. The mismatched synchronization may occurfor at least a portion of a duration when the video preview playscontinuously one or more times.

More than one video preview may be supported as well. For example, whena second video preview is received, the computing device can correlatethe second video preview with a second audio file. The second audio filecan be played when the second video preview is activated. In someexamples, the first and second video previews can correspond with acomposite of video previews and a composite audio rule. The compositeaudio rule can identify when audio associated with the first videopreview is muted and/or when the audio associated with the second videopreview is played after the composite of video previews is activated bythe computing device.

More than one audio file may be supported as well. For example, when twoor more audio files correspond with two or more video previews in acomposite of video previews, various methods of playing and/or mutingthe audio files are possible. For example, the audio files can besequentially played, one audio file can be muted while the second audiofile is played, the two audio files can be combined and playedsimultaneously, a separate audio file associated with the composite(e.g., a composite audio file) can play, or any other method of playingmultiple audio files with multiple videos.

In some embodiments, the audio may be configured to play or stop playing(e.g., by the computing device). For example, an audio with acorresponding video preview or composite of video previews is located atthe particular location on the display in the GUI. In another example,the audio may be played or stopped from playing when an activationdevice (e.g., finger, mouse pointer) touches or taps on a screen of thecomputing device. In some examples, the audio may also be configured tostop playing after a certain number of iterations.

III. Creation of a Video Preview with Audio

Embodiments can incorporate audio with video previews. For example, thecomputing device can create a video preview with audio by associating anaudio rule with an audio file and video preview, so that the audio andvideo play for a viewer according to the rule. The video preview andaudio file can be correlated to create an audio-visual file. The usercan create the video preview with audio in a GUI.

A. Creating a Video Preview Using a Link to an Audio File

FIG. 3 shows a flowchart illustrating a method of generating a videopreview with audio according to an embodiment of the present invention.

At block 310, a video preview can be received. In some examples, thevideo preview can have a video start and a video end (e.g., in aone-minute video, the video start is 0:00 and the video end is 1:00).The video preview can also be generated from a portion of a full video.For example, the video previews can include one or more images from afull video and/or one or more identifiers associated with content of thevideo previews (e.g., a video about baseball can have an identifier of“sports”).

In some embodiments, the video preview is generated from a full videoand then provided to a computing device. For example, the video server210 may transmit a full video to a user device 230 through a GUI. TheGUI can be configured to provide the user with the ability to indicate avideo portion of a full video (e.g., a request button that allows theuser device 230 to indicate a timestamp in the video preview). The userdevice 230 may identify a video portion of the full video by identifyinga start/end time or other identification provided by the GUI. Thestart/end of the full video identified by a third party server 240 oruser device 230 can be transmitted to the provider server 220 togenerate the video preview. For example, the user device 230 may selectthe start time and end time of the full video provided by the videoserver 210 using a button on the GUI. In another example, the userdevice 230 may activate a button provided locally at the user device(e.g., via a software application, “app”) and identify a start time andend time of the full video at the user device 230.

At block 320, an audio file can be received. The audio can have an audiostart and an audio stop (e.g., in a one-minute audio, the audio start is0:00 and the audio end is 1:00). The audio file can be combined with thevideo preview to obtain an audio-video file.

The audio can originate with several sources. In an embodiment, the usercan select a portion of the audio from the same video, so that the audiodirectly corresponds with the video (e.g., using a GUI provided by thecomputing device to select the audio source as the received videopreview). In an embodiment, the user can select a different portion ofthe audio of the same video. This can help avoid a garbled orunintelligible audio file if separate portions of the full video wereused to create the video preview. This may also allow the user select anexemplary portion of audio that may not correspond with an exemplaryportion of video so that both the audio and video can be representativeof the preview.

The user may select an external source of audio to incorporate with thevideo preview. For example, the user can provide a file location to anaudio source on a hard drive of the user device 230, external storagelocation (e.g., cloud storage), or URL to a storage service (e.g.,SoundCloud®). The user may upload audio to the storage location and/oredit the audio file. When more than one audio file is used (e.g., twoaudio files), the first audio file can come from a first source ortransmission (e.g., transmitted data packets) and the second audio filecan come from a second source or transmission. In some examples, themultiple audio files can come from the same source or transmission.

The audio file may be stored as a single file, multiple files, orencoded with the same file as the video preview. For example, if theaudio file is taken from an external source, the audio file can be alink to that external source and/or a duplicate copy of the audio filecan be stored with the computing device (e.g., provider server 220). Theaudio file may also be stored as a track or a standalone file on aserver.

At block 330, an audio rule can be received. The audio rule can specifyone or more criteria for how the audio file is to be played. Forexample, the audio rule can specify the criteria for how the audio fileis to be played when the video preview is played in response to anaudio-video object being activated. In another example, the audio rulecan specify when to alter the audio (e.g., fading the audio based on themovement of the audio-video object around a screen of a computingdevice).

For example, when the time that the audio file plays is longer than thetime that the video preview plays, the user can speed up the audio fileso that the audio file and video preview stop at the same time. Inanother example, the user may choose to repeat a portion of audio sothat the audio file and video preview start and/or stop at the sametime. In yet another example, the audio file ends and restarts beforethe video preview has completed at least once.

The audio rule can define when to repeat or stop the audio. For example,the audio file may not be the same duration as the video. The user canchoose to loop the video/audio to match the other. For example, theaudio file can be 60 seconds long and the video can be 10 seconds long.When looped, the same 10 second portion of video plays and repeats for aviewer six times, while a single 60 second clip of audio plays once.This may be useful when the video shows a repetitive task, like a personshoveling gravel into a wheelbarrow, where the action in the video isessentially repeated in real-life.

The audio can be altered (e.g., from its original state or from itsreceived condition). The alterations can be saved as part of the audiorule. For example, the user may alter the speed of the audio file. Forexample, the user may speed up or slow down the audio file. This mayhelp the audio file fit within a larger video preview that the user iscreating.

In another example, the user may add special effects to the audio file.For example, the user can combine two sources of audio together into oneaudio file. The user can also associate a Doppler Shift audio effect, sothat when a viewer is browsing through video previews, the pitch of theaudio can change and simulate the video preview moving farther away fromthe viewer (e.g., the pitch is lowered to simulate the video previewmoving farther away). This may also help avoid stuttering when the videopreview starts or stops. In yet another example, the audio ruleidentifies a volume of the audio file, so that when a viewer operatingthe computing device moves the video preview from the right of thedisplay to the center of the display, the pitch of the audio file ishigher according to the audio rule.

In another example, the user may also include a fade in or out of theaudio within a video preview. For example, the user may start with avideo preview and add a segment of audio from one part of the originalvideo, a segment of the audio from an external source, and then chooseto fade the audio in and/or out between the two segments. In anotherexample, the audio rule identifies a volume of the audio file, so thatwhen a viewer operating the computing device moves the video previewaway from a specified location of the display, the audio file fades outaccording to the audio rule.

At block 340, data of the audio-video file can be analyzed to determinewhich criteria is satisfied. For example, the audio rule may includerestrictions on when audio plays with a composite of video previewswhere the composite of video previews includes four videos. When theanalyzed video preview is a single video preview where one video previewcorresponds with one audio file, the criteria may not be satisfied. Inanother example, the audio rule can include criteria on which audio fileplays for a particular device based on an identifier. When theidentifier identifies a device with a small screen, the criteria cancorrelate with playing only one video preview at a time and one audiofile at a time.

At block 350, an audio-video object can be generated from the videopreview and audio file based on the audio rule. For example, the audiofile can be played according to the audio rule when the audio-videoobject is activated. The audio-video object can be activated in avariety of ways, including when the audio-video object is located at aparticular location.

B. GUI for Adding Audio

Embodiments can provide a graphical user interface where users canspecify a range in time in one or more video previews and audio files tocreate a video preview with audio. For example, the user may specify a5-second portion of a demolition video that shows when the buildingfalls to the ground and add a sound effect that corresponds to a soundof a crashing building. This 5-second portion may represent the bestanimated moment of the full video or the essence of the full video.

FIG. 4 shows a graphical user interface illustrating a video preview andone or more audio files according to an embodiment of the presentinvention. For example, the GUI 400 can include a video preview 410, oneor more depictions of the video preview and audio in an audio/videoeditor 420, a tool to provide a video 460, a tool to provide audio 470,a create tool 480, and a cancel tool 490. The audio/video editor 420 caninclude illustrations of the duration of the video preview 430, a firstaudio 440, and a second audio 450.

The GUI 400 can include a video preview 410. For example, the user mayprovide a path or link to a full video by activating (e.g., pressing,tapping) the tool to provide a video 460. The user can specify the pathto the video (e.g., a source at a client computer, at an external sourcelike a website, a provider server 220, or at a video server 210). Insome examples, the user can provide a full video and the computingdevice (e.g., provider server 220) can create a video preview anddisplay the video preview in the GUI 400.

The video preview may be uploaded to a computing device and identifiedby the GUI 400 using the tool to provide a video 460. In an embodiment,the video portion may be uploaded to the computing device directly(e.g., without a full video uploaded as well). The video portion can becombined with other previews, a group of images, or a full encoded videofile (such as an MP4 file) to use to generate a video preview (e.g., todisplay at 410).

The user can provide video using the tool to provide a video 460 and/orprovide audio using the tool to provide audio 470. For example, a usercould create a 5-minute audio file while watching Netflix® or HBO GO® inhis or her browser. The 5-minute file may not necessarily link to thehosted content (e.g., at the video server 210) and instead link to alocation of where the newly created 5-minute file is stored (e.g., atthe user device 230).

In an embodiment, the user may select multiple videos using the tool toprovide a video 460 in order to incorporate into one or more videopreviews. For example, the user can include “snippets” or small portionsof one or more full videos to generate a video preview (e.g., one videoportion from one television show and one video portion from anothertelevision show, or a combination of video portions to showcase aparticular actor in many roles). For example, one snippet can be takenfrom the video of one episode of the show and another snippet can betaken from a different episode of the show. As another example, onesnippet can be taken from one show on a first channel and anothersnippet can be taken from a different show on the first channel.

The user may also select multiple audio files using the tool to provideaudio 470 in order to incorporate into one or more audio files with thevideo previews. For example, the user can include “snippets” or smallportions of one or more full audio files to generate an audio file(e.g., one audio portion from one television show and one audio portionfrom another television show, or a combination of audio portions toshowcase a particular actor in many roles). For example, one snippet canbe taken from the audio of one episode of the show and another snippetcan be taken from a different episode of the show. As another example,one snippet can be taken from one show on a first channel and anothersnippet can be taken from a different show on the first channel.

The GUI 400 can also include an audio/video editor 420. The audio/videoeditor 420 can display a visual representation of a duration of thevideo preview 430, a duration of the first audio 440, and a duration ofthe second audio 450. The audio/video editor 420 can include thisinformation at any time (e.g., when the video preview is received by thecomputing device). The audio/video editor 420 can display one or morevideo previews or audio files associated with one or more video previewsin a temporal-based interface (e.g., the x-axis/horizontal-axis is theprogression of time in which the audio plays). The user may be able toview or adjust (e.g., dragging, tapping, clicking, sliding) a visualrepresentation of the audio in order to correlate the audio as the audioplays with the video preview (e.g., duration, pitch, volume, fading)through the use of the audio/video editor 420. In another example, thecomputing device can adjust the audio without instruction from the user.

Once a path or link to the video and audio have been provided, the usermay use the audio/video editor 420 to select the range of image framesfrom the video or audio or a range of time from the video or audio touse as a video preview object. In some embodiments, the user can selecta range of time (e.g., 10 seconds). In other embodiments, individualframes could be listed and a user could select a certain number offrames (e.g., 1-10) or select the individual frames (e.g., frames 0, 2,5, 10, 15, and 20). The range can be stored as a set of images in anyrelevant image format (e.g., GIF, JPG). The set of images or series ofnotes can be placed adjacent to each other (e.g., strung together) ineither the video preview or the audio file. This may create a videopreview object.

Once received at the GUI (e.g., provided by a computing device,including the provider server 220 or user device 230), the user can alsoadjust a video start, video stop, audio start, or audio stop using oneor more scroll bars provided by the audio/video editor 420. The scrollbars may be a representation of the video and audio in terms of time.For example, the length of the scroll bar can represent 4 minutes, whichcould be the entire length of the video.

Non-consecutive frames or images can be chosen as well. For example, auser can select several clips of an audio file that would playconsecutively. In one embodiment, the audio/video editor 420 can containa draggable “gripper” that enables the user to select a precise numberof sections to use for the audio file. Multiple grippers could be addedto the audio timeline and numbered. Additionally, an uploading interfacecould enable users to upload individual snippets of audio (e.g.,crashing car, pots banging) and splice the snippets together to createan audio file with non-consecutive frames.

The user may also correlate the audio to video by specifying when theaudio file should start and stop in relation to the video. For example,the user can move the visual representation of the audio start using the“gripper” to a location that lines up with the video start. In anotherexample, the audio/video editor 420 can accept a request by the user tocorrelate the audio to video. In yet another example, the audio andvideo may be automatically correlated by being provided to the GUI, suchthat the GUI generates an audio-video object from any uploaded contents.

The GUI 400 may also include a create tool 480 and a cancel tool 490.For example, once the user has selected the frames or time to use forthe video or audio, the user can select the create tool 480 to create avideo preview, audio file, or audio-video object, or the cancel tool 490to not create the same.

In some embodiments, the create tool 480 can be used to generate anaudio rule. For example, by activating the create tool 480, the audioand/or video identified in the GUI 400, any video start/end or audiostart/end, or alterations therein can be saved as the audio rule. Inanother example, if the audio start is not synchronized to start withthe video start in the audio/video editor 420, the audio rule may alsospecify that the audio start is not synchronized to start with the videostart.

IV. Audio Rules

Audio can play for different layouts or playable groups of videopreviews.

A. Default Audio Rules

A series of default audio rules can be identified or received inrelation to an audio-video object. For example, all audio for videopreviews can be muted or silenced when browsing. When the audio-videoobjects are activated, the video preview can open (e.g., in a newwindow) and begin to play video and audio automatically.

In some examples, a background audio can play when the user is browsingthrough the video previews. The background audio may not be associatedwith any one particular video preview. In a channel setting, thebackground audio can correspond with the channel instead of the videopreviews. For example, in a channel directed to cooking shows, thebackground audio can be a song about baking and the audio associatedwith the audio-video objects can be the cooking show host explaining aportion of the recipe.

In an embodiment, a default audio portion may be selected. For example,audio associated with a top block of a 1 block×2 block composite ofvideo previews can be playing. The viewer may be able to select anotherportion of the composite of video previews, e.g., the bottom block, toactivate different audio.

B. Audio Rules Based on Device

In some embodiments, the audio rules can be based on a device thatdisplays the audio-video objects (e.g., a user device 230). Thecomputing device (e.g., provider server 220) can receive an identifierrelating to a device. When the device is known to have a small screen(e.g., a small smartphone like the Apple® iPhone®), the audio rule canplay a single audio file. This may assume that only one video previewcan display at a time. In some embodiments, each video preview is laidout as a single playable group and the singly playable groups can beplayed in serial.

In another example, the identifier can identify a user device that isknown to have a larger screen (e.g., a medium-sized tablet computer likethe Apple® iPad®). In some embodiments, a composite audio can play. Theaudio rule (or composite audio rule) may not play the audio associatedwith the video previews in the playable group. In some embodiments, noaudio may play so that the browsing can be silent. The video previewscan play and audio can play when the playable group is activated (e.g.,tapped or moved to a particular location).

C. Audio Rules for Composite

FIG. 5 shows a graphical user interface illustrating a composite ofvideo previews according to an embodiment of the present invention. Forexample, the composite of video previews 500 can include one or morevideo previews (e.g., 510, 520, 530, and 540). The video previews can beorganized into one or more composites, including a first composite 550containing video preview 510 and video preview 520, and a secondcomposite containing video preview 530 and video preview 540.

Audio can play that corresponds with the composite of video previews.The audio may play while these previews are selected and/or activated,and the video associated with these groups of video previews areplaying. The audio associated with other composites may be silent. Whena user swipes or changes the focus, so that the other composites becomeactivated, the system can gauge, in a Doppler Shift-like audio effect,how the audio would change if an object was moving farther away from theviewer. For example, the pitch may change to seem like the firstcomposite 550 and second composite 560 are physically moving fartheraway from the viewer. The pitch may also change for the audio associatedwith other composites to simulate the videos physically moving closer tothe viewer.

A composite audio can play when the composite arrives at a particularlocation. For example, when a composite is moved to a center (or upperleft, right, or bottom) of the screen of a computing device (e.g., theuser device 230) or GUI, the video previews and/or audio associated withthe composite can play (e.g., according to the composite audio rule).

A composite audio is an entirely separate audio track. For example, thefirst video preview can include audio and the second video preview caninclude audio. The composite audio can be different audio than the audioassociated with the first or second video previews. In another example,the composite audio is one of the audio files from one of the videopreviews. The other audio files may not be played (e.g., associating thefirst audio file with the composite audio file and not playing thesecond audio file).

A combined audio can play for the composite. For example, each videopreview can be associated with audio and each of the audio tracks playat the same time (e.g., the first and second audio tracks playtogether). In another example, each video preview can be associated withaudio and each audio track can play sequentially (e.g., the first audiotrack plays first, the second audio track plays second, etc.).

A composite audio is one of the audio tracks associated with the videopreview, and that audio track plays for the composite of video previews.For example, the first audio file is associated with the first videopreview, the second audio file is associated with the second videopreview, and the composite audio file is associated with the compositeof video previews (e.g., both the first and second video previews). Thecomposite audio can play instead of the first audio or second audio.

The individual audio files associated with video previews can playsequentially. For example, three audio files associated with three videopreviews can be associated with each other in a composite. The firstaudio can play first, the second audio can play second, and third audiocan play third. In some embodiments, the audio can loop. For example,the audio can play more than once. In some examples, the audio files maynot need to be stored separately as a composite because the audio filesmay already be stored as audio files for the video previews.

V. Browsing Video Previews with Audio

Audio can correspond with video previews when a viewer is browsingthrough the video previews. For example, the user can select the videopreview and the audio can start playing, when audio is associated withthe video preview. Audio may start playing after additional actions aswell (e.g., an additional click or tap on a selected video preview).

Audio may also correspond with the movement of the video preview withrespect to the video preview's position in the channel or GUI. Forexample, as the video preview is scrolled off the screen, the pitch ofthe audio can change so that the video preview physically seems like itis moving farther away. This may be implemented in an embodiment thatuses the Doppler Shift audio effect.

In an embodiment, the audio may fade in and out. For example, when afirst video preview is selected and playing for a viewer, the audio maybe playing at full volume. Once the user swipes or selects a secondvideo preview, the audio associated with the first video preview can belowered, the video preview can move and/or become de-selected, and thesecond video preview can be selected. In an embodiment, once the secondvideo preview is selected, the audio associated with the video previewcan become louder until the audio associated with this second videopreview is at full volume. This can help avoid a choppy sounding audio.In other embodiments, the audio may fade in and out at different speeds.The amount of silence (e.g., lack of audio) may also be altered.

A. Browsing Composite Video Previews with Audio

FIG. 6 shows a flowchart illustrating a method of browsing videopreviews with audio according to an embodiment of the present invention.

At block 610, a composite audio-video object can be received. Thecomposite audio-video object can include video preview(s) (e.g., a firstvideo preview and second video preview), a composite audio file, and acomposite audio rule. For example, the composite audio file may beassociated with the composite audio-video object and the composite audiofile can be played according to a composite audio rule when thecomposite of video previews is activated.

In some embodiments, the composite can correspond to playing multiplepreviews and/or multiple audio files (e.g., the composite includesmultiple blocks, each playing a preview). The video previews in acomposite can each link to the same full video. In one embodiment, thecreator of a preview can identify the previews or audio files to includein a composite, and the composite can exist as a single display object,where the previews of the composite start playing at the time thecomposite is activated. The shape of a composite of video previews canbe a square (2 blocks×2 blocks, N blocks×N blocks) or rectangle (1×N,2×N, 2×2 with blocks comprising unequal sides, N×N with blockscomprising unequal sides).

The composite audio file can have an audio start and an audio stop(e.g., in a one-minute audio, the audio start is 0:00 and the audio endis 1:00). In some embodiments, the composite audio file can include oneor more audio files associated with one or more video previews. In otherembodiments, the composite audio file is associated with the compositeas a whole. The composite audio file may be stored as a single file,multiple files, or encoded with the same file as the composite of videopreviews. For example, if the composite audio file is taken from anexternal source, the composite audio file can be a link to that externalsource and/or a duplicate copy of the composite audio file can be storedwith the computing device (e.g., provider server 220). The compositeaudio file may also be stored as a track or a standalone file on aserver.

In some embodiments, the composite audio-video object can be received bya computing device (e.g., user device 230) through a GUI (e.g., anetwork page, an application, etc.). The GUI can display the compositeaudio-video object on a screen of the computing device.

At block 620, the composite audio-video object can be identified asactivated. For example, a computing device (e.g., user device) canidentify that a composite audio-video object is located at a particularlocation. For example, the computing device may identify that thecomposite audio-video object has moved to a particular location on thedisplay in the GUI. The computing device may identify the currentlocation of the composite audio-video object (e.g., pixel location,relative window location) and compare the current frame location withthe particular location (e.g., as defined within the GUI, defined by thevideo preview, or defined within the composite audio-video object usingdata obtained from the GUI or a server). When the composite audio-videoobject is placed in a particular location (e.g., the middle of a browserwindow, 10 pixels from the edge of the screen, in screen), the compositeaudio-video object can be activated.

At block 630, the video preview(s) and composite audio file can besimultaneously played according to the composite audio rule. Forexample, the video preview(s) and composite audio file can be played inresponse to the activation of the composite audio-video object. Thefirst video preview, the second video preview, and the composite audiofile may play according to the composite audio rule.

When the audio-video object is activated, the audio file may not beplayed for at least a portion of the duration when the video preview isplayed. For example, the composite audio file can play when thecomposite of video previews is activated, independent of activating thefirst video preview or the second video preview. In another example, theduration of the video preview can be two minutes. The audio-video objectcan be activated and the video preview can provide a series of images(e.g., that correspond with one or more portions in the full video)without audio. In some embodiments, the audio file may begin playing(e.g., after a few seconds) or may not play at all.

Playing the audio file may not be synchronized with the playing of thevideo preview. For example, the audio file can include an audio startand an audio end and the video preview can include a video start and avideo end. The audio file may not be synchronized with the playing ofthe video preview for at least a portion of a duration when the videopreview plays continuously one or more times. In another example, theaudio file can start at 0:10 minutes of a 2-minute video preview. Theaudio file may end at 0:50 minutes of the video preview and start over(e.g., loop or replay), or a second audio file can begin playing. Insome embodiments, the audio start and video start can correspond withthe same time (e.g., 0:00 of the video preview). In some embodiments,the audio end and video end can correspond with the same time (e.g.,2:00 of a 2-minute video preview).

In an embodiment, the composite audio file is looped to playcontinuously while the first video preview and the second video areplayed (e.g., because the duration of the composite audio file isshorter than the duration of the first video preview). For example, theaudio file can be 60 seconds long and the video can be 10 seconds long.When looped, the same 10 second portion of video plays and repeats for aviewer six times, while a single 60 second clip of audio plays once.

In an embodiment, the composite audio file is looped using othermethods. For example, the composite audio file can be looped to playcontinuously according to a composite audio rule or looped to playcontinuously while the composite audio-video object is active. Forexample, while the composite of video previews is located at aparticular location on a screen of a computing device, the videopreview(s) and audio file(s) can play. Other objects may not play (e.g.,sound or video).

In an embodiment, the composite audio file is not played for at least aportion of the time that the first video preview is playing. Forexample, the composite audio rule can identify an audio start as 0:10 ofthe one-minute video preview, so that the corresponding audio is notplayed for the first ten seconds.

B. Browsing Video Previews that Use Non-Consecutive Frames

FIG. 7 shows a flowchart illustrating a method of browsing videopreviews with audio according to an embodiment of the present invention.

At block 710, an audio-video object can be received. The audio-videoobject can include a video preview corresponding to non-consecutiveframes of a full video and an audio file. The audio file may be a fullaudio file from a full video. In some examples, the audio-video objectcan be received from a computing device (e.g., a provider server 220).The computing device may also create the audio-video object and providethe object to another computing device (e.g., a user device 230).

The video preview can correspond to non-consecutive frames of a fullvideo. For example, the full video can include ten frames total (e.g.,at one frame per second, the full video would be ten seconds long).Consecutive frames can include frames 1-4 and non-consecutive frames caninclude frames 1, 2, 5, 6, and 10. The non-consecutive frames can beused to create a video preview. The video preview can contain the samebit rate or a faster bit rate to create a video preview that is ashorter duration than the full video. More or fewer frames can be usedwithout diverting from the essence of the invention.

In some embodiments, a video preview can be non-consecutive frames andthe audio for the non-consecutive frames can correspond with other partsof the full video (e.g., not corresponding to the selected frames orimages). For example, the video preview can correspond with a buildingcrashing to the ground and the audio can include someone explaining whatthe building will look like when it crumbles. In another example, theaudio may include an unrelated audio file from another full video oraudio file. In yet another example, audio can be for frames 1-100, andonly frames 80-100 are used in the video preview.

At block 720, an activation of the audio-video object can be received.For example, the audio-video object can be activated based on the objectmoving to a particular location in the screen of the computing device(e.g., user device 230). In another example, the audio-video object maybe activated when a GUI is initially provided at the computing device.

At block 730, the video preview and audio file can be played. In someembodiments, the audio file does not correspond with the audio of thenon-consecutive frames. For example, the audio file is from the fullaudio file and does not correspond to the audio of the non-consecutiveframes of the video preview.

C. GUI for Displaying and Browsing Video Previews

FIG. 8 shows a graphical user interface illustrating an application thatdisplays video previews with audio according to an embodiment of thepresent invention. The GUI 800 can include one or more video previews810, 820, 830 and corresponding audio.

The audio may correlate with the video preview, such that video preview810 corresponds with a first audio 850, video preview 820 correspondswith a second audio 860, and video preview 830 corresponds with a thirdaudio 870. Alternatively, a general audio file 880 can correspond withthe plurality of video previews. In some embodiments, GUI 840 may bedisplayed when the audio-video object is created but not when the userbrowses the video previews with audio.

When audio corresponds with a particular video preview, the location ofthe video preview may affect the audio (e.g., pitch, volume, fading,Doppler shift). For example, the audio may play while these previews areselected and/or activated and the video associated with these groups ofvideo previews are playing. The audio associated with other compositesmay be silent. When a user swipes or changes the focus (e.g., clicks onan object displayed by the GUI 800 other than the video preview) so thatother video previews and/or frame objects become activated, the systemcan gauge, in a Doppler Shift-like audio effect, how the audio wouldchange if an object was moving farther away from the user.

In another example, the pitch may change to seem like one video previewis physically moving farther away from the user. For example, when firstaudio 850 corresponding with video preview 820 is moved from onelocation to another location (e.g., from the center location to the farleft location, partially off the screen), the pitch for first audio 850may be adjusted to simulate that the source of the audio is movingfarther from the user. In another example, when second audio 860corresponding to video preview 810 is moved from one location to anotherlocation (e.g., from far left location to the center location), thepitch for second audio 860 may be adjusted to simulate that the sourceof the audio is moving closer to the user. The pitch may also change forthe audio associated with other composites to simulate the videosphysically moving closer to the user.

In some embodiments, the audio may fade in or out. For example, theremay be one video preview associated with one audio on a GUI. When thevideo preview appears on the screen (e.g., moving from right to left,video preview 810), the audio can fade in as the video preview movescloser to the center of the screen. The audio may also fade out thefarther the video preview moves to the center of the screen (e.g., nearthe location of video preview 830). In another example, in response toidentifying that the frame object is located at the particular locationon the display in the GUI of the computing device, the audio can befaded in or out, irrespective of the location of the video preview tothe center of the screen (e.g., top of the screen to bottom of thescreen, closer or farther from the corner of the screen).

When more than one video preview is identified on a GUI, the locationsof both video previews can affect the audio. For example, in response toidentifying that the first frame object is located at the particularlocation on the display in the GUI. A first audio file (that correspondswith the first video preview) can fade out and a second audio file (thatcorresponds with the second video preview) can fade in.

VI. Example Subsystems and Components

Any of the clients or servers may utilize any suitable number ofsubsystems. Examples of such subsystems or components are shown in FIG.9. The subsystems shown in FIG. 9 are interconnected via a system bus975. Additional subsystems such as a printer 974, keyboard 978, fixeddisk 979, monitor 976, which is coupled to display adapter 982, andothers are shown. Peripherals and input/output (I/O) devices, whichcouple to I/O controller 971, can be connected to the computer system byany number of means known in the art, such as input/output (I/O) port977 (e.g., USB, FireWire®) For example, I/O port 977 or externalinterface 981 (e.g. Ethernet, Wi-Fi, etc.) can be used to connect thecomputer apparatus to a wide area network such as the Internet, a mouseinput device, or a scanner. The interconnection via system bus allowsthe central processor 973, which may include one or more processors, tocommunicate with each subsystem and to control the execution ofinstructions from system memory 972 or the fixed disk 979 (such as ahard drive or optical disk), as well as the exchange of informationbetween subsystems. The system memory 972 and/or the fixed disk 979 mayembody a computer readable medium. Any of the data mentioned herein canbe output from one component to another component and can be output tothe user.

It should be understood that any of the embodiments of the presentinvention can be implemented in the form of control logic using hardware(e.g. an application specific integrated circuit or field programmablegate array) and/or using computer software with a generally programmableprocessor in a modular or integrated manner. As user herein, a processorincludes a multi-core processor on a same integrated chip, or multipleprocessing units on a single circuit board or networked. Based on thedisclosure and teachings provided herein, a person of ordinary skill inthe art will know and appreciate other ways and/or methods to implementembodiments of the present invention using hardware and a combination ofhardware and software.

Any of the software components or functions described in thisapplication may be implemented as software code to be executed by aprocessor using any suitable computer language such as, for example,Java®, C++ or Perl using, for example, conventional or object-orientedtechniques. The software code may be stored as a series of instructionsor commands on a computer readable medium for storage and/ortransmission, suitable media include random access memory (RAM), a readonly memory (ROM), a magnetic medium such as a hard-drive or a floppydisk, or an optical medium such as a compact disk (CD) or DVD (digitalversatile disk), flash memory, and the like. The computer readablemedium may be any combination of such storage or transmission devices.

Such programs may also be encoded and transmitted using carrier signalsadapted for transmission via wired, optical, and/or wireless networksconforming to a variety of protocols, including the Internet. As such, acomputer readable medium according to an embodiment of the presentinvention may be created using a data signal encoded with such programs.Computer readable media encoded with the program code may be packagedwith a compatible device or provided separately from other devices(e.g., via Internet download). Any such computer readable medium mayreside on or within a single computer program product (e.g. a harddrive, a CD, or an entire computer system), and may be present on orwithin different computer program products within a system or network. Acomputer system may include a monitor, printer, or other suitabledisplay for providing any of the results mentioned herein to a user.

Any of the methods described herein may be totally or partiallyperformed with a computer system including one or more processors, whichcan be configured to perform the steps. Thus, embodiments can bedirected to computer systems configured to perform the steps of any ofthe methods described herein, potentially with different componentsperforming a respective steps or a respective group of steps. Althoughpresented as numbered steps, steps of methods herein can be performed ata same time or in a different order. Additionally, portions of thesesteps may be used with portions of other steps from other methods. Also,all or portions of a step may be optional. Additionally, any of thesteps of any of the methods can be performed with modules, circuits, orother means for performing these steps.

The specific details of particular embodiments may be combined in anysuitable manner without departing from the spirit and scope ofembodiments of the invention. However, other embodiments of theinvention may be directed to specific embodiments relating to eachindividual aspect, or specific combinations of these individual aspects.

The above description of exemplary embodiments of the invention has beenpresented for the purposes of illustration and description. It is notintended to be exhaustive or to limit the invention to the precise formdescribed, and many modifications and variations are possible in lightof the teaching above. The embodiments were chosen and described inorder to best explain the principles of the invention and its practicalapplications to thereby enable others skilled in the art to best utilizethe invention in various embodiments and with various modifications asare suited to the particular use contemplated.

A recitation of “a”, “an” or “the” is intended to mean “one or more”unless specifically indicated to the contrary.

1.-20. (canceled)
 21. A method comprising: receiving an audio file by acomputing device, wherein the audio file comprises an audio stop;receiving a video file by the computing device, wherein the video filecomprises a video stop; altering, by the computing device, the audiofile according to an audio rule, wherein the audio rule specifieslooping audio of the audio file to correlate the audio stop with thevideo stop; and generating an audio-video object that combines the audiofile and the video file, wherein the audio file and the video file areplayed according to the audio rule when the audio-video object isactivated at a screen of a playback device.
 22. The method of claim 21,wherein the audio rule also specifies fading the audio file.
 23. Themethod of claim 22, wherein the audio rule fades audio of the audio filebased on movement of the audio-video object on the screen of theplayback device.
 24. The method of claim 21, wherein the audio file alsospecifies an audio start and the video file also specifies a videostart.
 25. The method of claim 24, wherein the audio start and the videostart are synchronized to play the audio file and video filesimultaneously when the audio-video object is activated.
 26. The methodof claim 21, wherein a first duration of the audio file is shorter thana second duration of the video file and greater than zero.
 27. Themethod of claim 21, wherein looping the audio plays the audio filecontinuously while the video file is played.
 28. A computer productcomprising a non-transitory computer readable medium storing a pluralityof instructions that when executed control a playback device to play avideo file with audio, the instructions comprising: receiving an audiofile, wherein the audio file comprises an audio stop; receiving thevideo file, wherein the video file comprises a video stop; altering theaudio file according to an audio rule, wherein the audio rule specifieslooping audio of the audio file to correlate the audio stop with thevideo stop; and generating an audio-video object that combines the audiofile and the video file, wherein the audio file and the video file areplayed according to the audio rule when the audio-video object isactivated at a screen of the playback device.
 29. The computer productof claim 28, wherein the audio rule also specifies fading the audiofile.
 30. The computer product of claim 29, wherein the audio rule fadesaudio of the audio file based on movement of the audio-video object onthe screen of the playback device.
 31. The computer product of claim 28,wherein the audio file also specifies an audio start and the video filealso specifies a video start.
 32. The computer product of claim 31,wherein the audio start and the video start are synchronized to play theaudio file and video file simultaneously when the audio-video object isactivated.
 33. The computer product of claim 28, wherein a firstduration of the audio file is shorter than a second duration of thevideo file and greater than zero.
 34. The computer product of claim 28,wherein looping the audio plays the audio file continuously while thevideo file is played.
 35. A system comprising: one or more processorsconfigured to: receive an audio file, wherein the audio file comprisesan audio stop; receive a video file, wherein the video file comprises avideo stop; alter the audio file according to an audio rule, wherein theaudio rule specifies looping audio of the audio file to correlate theaudio stop with the video stop; and generate an audio-video object thatcombines the audio file and the video file, wherein the audio file andthe video file are played according to the audio rule when theaudio-video object is activated at a screen of a playback device. 36.The system of claim 35, wherein the audio rule also specifies fading theaudio file.
 37. The system of claim 36, wherein the audio rule fadesaudio of the audio file based on movement of the audio-video object onthe screen of the playback device.
 38. The system of claim 35, whereinthe audio file also specifies an audio start and the video file alsospecifies a video start.
 39. The system of claim 38, wherein the audiostart and the video start are synchronized to play the audio file andvideo file simultaneously when the audio-video object is activated. 40.The system of claim 35, wherein a first duration of the audio file isshorter than a second duration of the video file and greater than zero.41. The system of claim 35, wherein looping the audio plays the audiofile continuously while the video file is played.